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Preventing and Treating Youth Marijuana Use: 
An Updated Review of the Evidence 


Initiative 502 (I-502) legalized recreational 


marijuana use for adults in Washington State. 


The law directs the Washington State 
Institute for Public Policy (WSIPP) to 
conduct a benefit-cost evaluation of the 
implementation of I-502." 


The initiative also requires the Division of 
Behavioral Health and Recovery (DBHR) in 
the Department of Social and Health Services 
(DSHS) to allocate at least 85% of its share of 
disbursements from the state’s dedicated 
marijuana fund to “evidence-based and cost- 
beneficial programs and practices” to 
prevent and treat use among middle and 
high school-aged youth.” 


In September 2014, to help DBHR meet this 
requirement and as part of our broader 
I-502 assignment, WSIPP published an 
independent benefit-cost analysis of 13 
youth prevention programs from DBHR’s 
preliminary list of evidence-based programs 
with marijuana prevention outcomes.’ 


' The initiative requires a progress report by September 2015 
and the final outcome evaluation by 2032. 

* Initiative Measure No. 502. Full text available at 
http://apps.leg.wa.gov/documents/billdocs/2011- 
12/Pdf/initiatives/Initiatives/INITIATIVE%20502.pdf 

2 Hanley, S., & Aos, S. (2014). Preventing youth 

substance use: A review of thirteen programs. (Doc. No. 14-09- 
3201). Olympia: Washington State Institute for Public Policy. 











Summary 


Initiative 502, passed by Washington voters in 
November 2012, legalized recreational marijuana 
use for adults in the state. The initiative directs 
WSIPP to evaluate the policy in a series of reports 
between 2015 and 2032. 


As part of this assignment, we reviewed the 
evaluation literature on 23 youth marijuana 


prevention and treatment programs. The evidence 
indicates that 18 of the 23 programs have, on 
average, benefits that are greater than costs. 


The legislature has established a three-tiered 
classification for programs: “evidence-based,” 
“research-based,” and “promising.” Using these 
definitions, three of the reviewed programs are 
evidence-based, 14 are research-based, and two 
are promising. 


In this update, we analyze ten additional 
prevention and treatment programs that 
have been studied regarding their impacts 
on youth marijuana use. We apply WSIPP’s 
standard research methodology to 
determine whether—and to what degree of 
certainty—each program's benefits are likely 
to exceed costs. Our methodology involves a 
three-step procedure, described in the next 
section. 


Suggested citation: Lemon, M., Pennucci, A., Hanley, S., & Aos, S. 
(2014). Preventing and treating youth marijuana use: An updated 


review of the evidence. (Doc. No. 14-10-3201). Olympia: 
Washington State Institute for Public Policy. 





I. Research Approach 


When WSIPP carries out study assignments 
from the legislature to identify what works in 
public policy, we implement a three-step 
research approach. 


Step 1: What Works? What Does Not? 





In the first step, we estimate whether various 
programs and policies can achieve desired 
outcomes, such as reductions in youth 
marijuana use. We carefully analyze all high- 
quality studies from the United States and 
elsewhere to identify those programs and 
policies found to impact outcomes. We focus 
on research studies with strong evaluation 
designs and exclude studies with weak 
research methods. 


Our empirical approach follows a meta- 
analytic framework to systematically assess 
all credible evaluations we can locate ona 
given topic. Given the weight of the 
collective evidence, we calculate an average 
expected effect of a program or policy ona 
particular outcome of interest—in this case, 
marijuana use. 


In our analyses, we consider the programs’ 
effects on all reported outcomes. When we 
reviewed the evaluation literature for these 
23 programs and conducted the meta- 
analyses, we examined their effects on 
marijuana use as well as alcohol, tobacco, 
and other drug use; mental health (e.g., 
anxiety and depression); school-related 
outcomes such as test scores and high 
school graduation; and crime. Examining 
these additional outcomes allows us to get 
a comprehensive view of effectiveness and 
provide better estimates of the overall 
benefits and costs that can be expected 
from statewide implementation. 





Step 2: What Makes Economic Sense? 


Next, we consider the costs and benefits of 
implementing the program or policy by 
answering two questions: 


e How much would it cost Washington 
taxpayers to produce the results 
found in Step 1? 


e How much would it be worth to 
people in Washington State to 
achieve the results found in Step 1? 


That is, in dollars and cents, what are the 
costs and benefits of each program or 
policy? 


To answer these questions, we have 
developed, and continue to refine, an 
economic model that estimates benefits 
and costs. The model provides an internally 
consistent monetary valuation so program 
and policy options can be compared on an 
apples-to-apples basis. Our benefit-cost 
results include standard financial statistics: 
net present values and benefit-cost ratios. 


We present monetary estimates from three 
perspectives: 

a) program participants, 

b) taxpayers, and 

c) other people in society. 
The sum of the three perspectives provides a 
“total Washington” view on whether a 


program or policy produces benefits that 
exceed costs. 


Step 3: What is the Risk in the Benefit-Cost 
Findings? 





Any tabulation of benefits and costs involves 
a degree of risk about the estimates 
calculated. This is expected in any investment 
analysis, whether in the private or public 
sector. To assess the riskiness of our 
conclusions, we perform a “Monte Carlo 
simulation” in which we vary key factors in 
our calculations. The purpose of this analysis 
is to determine the probability that a 
particular program or policy will at least 
break even. 


Thus, we produce two “big picture” findings 
for each program: an expected benefit-cost 
result and, given our understanding of the 
risks, the probability that the program will at 
least have benefits that are greater than 
costs. 


Brief descriptions of the 23 programs 
contained in this report can be found in the 
Appendix and on our website.” Readers 
interested in an in-depth description of our 
research methods can review our Technical 
Documentation.” 





* http://www.wsipp.wa.gov/BenefitCost 

° Washington State Institute for Public Policy, (2014). Benefit- 
cost technical documentation. Olympia, WA: Author. Available 
online at 
http://www.wsipp.wa.gov/TechnicalDocumentation/WsippBen 
efitCostTechnicalDocumentation.pdf 











II. Findings 


The findings from our benefit-cost analyses are 
presented in Exhibit 1 (next page).° Of the 21 
programs for which we were able to calculate 
benefits and costs, 18 have a favorable result.’ 
That is, 18 of the programs have, on average, 
benefits that exceed costs. 


In Exhibit 1, we also present the second key 
bottom-line finding for each program: the odds 
that benefits will exceed costs, after we take into 
account the uncertainty in our estimates. The 
higher the odds, the more confident we are that 
benefits will, in fact, outweigh cost. For five of 21 
programs reviewed, benefits exceed costs at 
least 75% of the time. 


The legislature has established a three-tiered 
classification for programs: evidence-based, 
research-based, and promising (see sidebar). ® 
Using these definitions, three programs—Life 
Skills Training, Communities that Care, and Teen 
Marijuana Check-Up—have a sufficiently high 
probability to establish them as top-tier 
evidence-based programs. Fourteen programs 
meet the criteria for second-tier research-based 
programs, while two programs are promising. 
Two programs (InShape and Project SUCCESS) 
produce poor outcomes. Exhibit 2 (on page 6) 
summarizes the classifications. 


° For detailed results on each program, please see 
http://www.wsipp.wa.gov/ReportFile/1572/Wsipp Preventing- 
and-Treating-Youth-Marijuana-Use-An-Updated-Review-of-the- 
Evidence Benefit-Cost-Results.pdf 

7 We did not find any evaluations that met our criteria for two 
programs on DBHR’s preliminary list (Project Venture and Red 
Cliff Wellness School Curriculum). As a result, we cannot 
determine the potential effectiveness of these two programs, and 
they are not included in our benefit-cost results. 

8 RCW 71.24.025. Since 2012, WSIPP has been directed by the 
legislature to create evidence-based, research-based, and 
promising program inventories for adult behavioral health, child 
welfare, children’s mental health services, juvenile justice, adult 
corrections, and the K-12 Learning Assistance Program. 











Legislative Definitions of Evidence-Based, 
Research-Based, and Promising Practices 
Contained in RCW 71.24.025 


Evidence-based 


A program or practice that has been tested 
in heterogeneous or intended populations 
with multiple randomized, or statistically 
controlled evaluations, or both; or one large 
multiple site randomized, or statistically 
controlled evaluation, or both, where the 
weight of the evidence from a systemic 
review demonstrates sustained 
improvements in at least one outcome. 
"Evidence-based" also means a program or 
practice that can be implemented with a set 
of procedures to allow successful replication 
in Washington and, when possible, is 
determined to be cost-beneficial. 


Research-based 


A program or practice that has been tested 
with a single randomized, or statistically 
controlled evaluation, or both, 
demonstrating sustained desirable outcomes; 
or where the weight of the evidence from a 
systemic review supports sustained outcomes 
[.. .] but does not meet the full criteria for 
evidence-based. 


Promising 


A practice that, based on statistical analyses 
or a well-established theory of change, 
shows potential for meeting the evidence- 
based or research-based criteria [. . .]. 


Exhibit 1 
Benefits and Costs for Programs to Prevent or Treat Youth Marijuana Use 






















































































| ne Benefits eee: 
— Non- minus ; sofa 
Program name Benefits ‘benetts ‘exPayer Costs costs (net Coctratig,_Postive net 
value) 

Prevention Programs 
Life Skills Training $3,461 $804 $2,657 ($97) | $3,363 $35.66 93% 
Communities that Care $2,079 $626 $1,453 ($574) | $1,505 $3.70 85% 
Project STAR $3,917 $941 $2,977 ($499) | $3,418 $7.86 84% 
Lions Quest Skills for Adolescence $461 $89 $372 ($95) $366 $4.88 75% 
SPORT $1,339 $308 $1,030 ($38) | $1,300 $34.84 73% 
keepin’ it REAL $813 $244 $569 ($48) $765 $16.98 73% 
Strengthening Families for Parents and youth 10-14. $4,259 $1,061 $3,197 ($1,098) | $3,160 $3.89 70% 
Multidimensional Treatment Foster Care $17,286 $4,256 $13,031 ($8,111) | $9,175 $2.13 67% 
Case management in schools $5,252 $1479 $3,773 ($248) $5,005 $21.21 66% 
Project Northland $717 $184 $533 ($185) $532 $3.87 65% 
ee the Drug Free Years) $1,951 $653 | $1,298 ($655) | $1,296 $2.99 ied 
Caring School Community (formerly Child $8,611 $2,171 $6,440 ($1,218) | $7,393 $7.06 62% 
Development Project) 
aac erm = be al $10,694 $3,513 $7,181 ($3,193) | $7,501 $3.36 60% 
Project ALERT $331 $119 $213 ($147) $184 $2.25 55% 
Project Towards No Drug Abuse (TND) $174 $44 $130 ($64) $110 $2.73 51% 
eee ee Family Support) ie ee a0 | eee) | eae ae ae 
InShape ($309) ($90) ($219) ($15) | ($324) ($21.00) 47% 
Project SUCCESS ($209) ($20) ($189) ($155) ($364) | ($1.35) 46% 
Treatment Programs 
Teen Marijuana Check-Up $1,898 $588 $1,310 ($106) $1,793 $17.94 100% 
Adolescent Assertive Continuing Care $11,089 $3,387 $7,702 ($2,181) | $8,907 $5.09 68% 
be teeter a UnSEepy MOET jitor $14,292 $4,308 $9,984 ($7,804) $6,488 $1.84 67% 





























Exhibit 2 
Inventory of Evidence- and Research-Based Practices: 
Prevention and Treatment of Youth Marijuana Use 


2 Reason program does not meet 
i Level of Benefit-cost i ee 
Program/intervention , evidence-based criteria 
evidence percentage mee 
(see full definitions below) 


Percent 
minority 


Prevention Programs 
Life Skills Training 
Communities that Care 


Project STAR 

Lions Quest Skills for Adolescence 

keeping’ it REAL 

SPORT 

Strengthening Families for Parents and Youth 10-14 
Multidimensional Treatment Foster Care 

Case management in schools 

Project Northland 

Guiding Good Choices (formerly Preparing for the Drug Free Years) 
Mentoring for students: Community-based (including volunteer costs) 
Project ALERT 

Family Check-Up (also known as Positive Family Support) 

Caring School Community (formerly Child Development Project) 
Project Towards No Drug Abuse 


InShape 
Project SUCCESS 
Project Venture 
Red Cliff Wellness School Curriculum 
Treatment Programs 
Teen Marijuana Check-Up 
Adolescent Assertive Continuing Care 
Multidimensional Family Therapy (MDFT) for substance abusers 


Heterogeneity 
Single evaluation 
Benefit-cost 
Single evaluation/Benefit-cost 
Single Evaluation/Benefit-cost/Heterogeneity 
Benefit-cost/Heterogeneity 
Mixed results/Benefit-cost 
Benefit-cost 
Benefit-cost/Heterogeneity 
Mixed results/Benefit-cost 
Benefit-cost/Heterogeneity 
Benefit-cost 
Weight of evidence/Benefit-cost 
Mixed results/Weight of evidence/Benefit-cost 
Produces poor or null outcomes 
Produces poor or null outcomes 
Research on outcomes of interest not yet available 
Research on outcomes of interest not yet available 
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Benefit-cost/Heterogeneity 
Benefit-cost 
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Key: 

@ Evidence-based 

© Research-based 

© Produces null or poor outcomes 
P_ Promising 

NR_ Not reported 


Reasons Programs May Not Meet Suggested Evidence-Based Criteria: 

Benefit-cost: The WSIPP benefit-cost model was used to determine whether a program meets this criterion. Programs that do not achieve at least a 75% chance of positive 
net present value do not meet the benefit-cost test. 

Heterogeneity: To be designated as evidence-based, a program must have been tested on a “heterogeneous” population. We operationalized heterogeneity in two ways. 
First, the proportion of minority program participants must be greater than or equal to the minority proportion of children in Washington State aged 0 to 17. From the 2010 
Census, for children aged 0 through 17 in Washington, 68% were white and 32% minority. Thus, if the weighted average of program participants had at least 32% minorities 
then the program was considered to have been tested on a heterogeneous population. Second, the heterogeneity criterion can also be achieved if at least one of the studies 
has been conducted on youth in Washington and a subgroup analysis demonstrates the program is effective for minorities (p < .20). Programs passing the second test are 
marked with a “. Programs that do not meet either of these two criteria do not meet the heterogeneity definition. 

Mixed results: If findings are mixed from different measures (e.g., undesirable outcomes for behavior measures and desirable outcomes for test scores), the program does 
not meet evidence-based criteria. 

Program cost: A program cost was not available to WSIPP at the time of the inventory. Thus, WSIPP could not conduct a benefit-cost analysis. 

Research on outcomes of interest not yet available: The program has not yet been tested with a rigorous outcome evaluation. 

Single evaluation: The program does not meet the minimum standard of multiple evaluations or one large multiple-site evaluation. 





Weight of evidence: Results from a random effects meta-analysis (p 2 .20) indicate that the weight of the evidence does not support desired outcomes, or results from a 
single large study indicate the program is not effective. 

Level of Evidence: 

Evidence-based: A program or practice that has been tested in heterogeneous or intended populations with multiple randomized and/or statistically-controlled evaluations, 
or one large multiple-site randomized and/or statistically-controlled evaluation, where the weight of the evidence from a systematic review demonstrates sustained 
improvements in at least one outcome. Further, “evidence-based” means a program or practice that can be implemented with a set of procedures to allow successful 
replication in Washington and, when possible, has been determined to be cost-beneficial. 

Research-based: A program or practice that has been tested with a single randomized and/or statistically-controlled evaluation demonstrating sustained desirable 
outcomes; or where the weight of the evidence from a systematic review supports sustained outcomes as identified in the term “evidence-based” in RCW (the above 
definition) but does not meet the full criteria for “evidence-based.” 

Promising practice: A program or practice that, based on statistical analyses or a well-established theory of change, shows potential for meeting the “evidence-based” or 
“research-based” criteria, which could include the use of a program that is evidence-based for outcomes other than the alternative use. 

Other Definitions: 

Benefit-cost percentage: The percent of the time monetary benefits exceed costs. 





Appendix 


Preventing and Treating Youth Marijuana Use: An Updated Review of the Evidence 


This appendix provides brief summaries of the programs reviewed in the main report. 


Adolescent Assertive Continuing Care 





This intervention was designed for youth returning to the community after residential substance 
abuse treatment. The aim of the intervention is to encourage youth to continue in outpatient 
treatment. Caseworkers make weekly home visits, advocate for needed services, and aid in job 
search and other pro-social activities. 


Caring School Community (formerly Child Development Project) 


Caring School Community is a whole-school program aimed at promoting positive youth 
development. Designed for elementary schools, the program attempts to promote prosocial 
values, improve academic achievement, and prevent drug use, violence, and delinquency by 
encouraging collaboration among students, staff, and parents. 


Case management in schools 





Case management involves placing a full-time social worker or counselor in a school to help 
identify at-risk students’ needs and connect students and families with relevant services in and 
outside of the K-12 system. Three such models have been evaluated and are included in this 
analysis (in no particular order): Communities in Schools, City Connects, and Comer School 
Development Program. In practice, each of these models includes other services (such as 
extended learning time and educator training), but the program evaluations focus on the impact 
of the case management component. 


Communities that Care 





Communities that Care (CTC) is a coalition-based community prevention program that aims to 
prevent youth problem behaviors including underage drinking, tobacco use, violence, 
delinquency, school dropout, and substance abuse. CTC works through a community board to 
assess risk and protective factors among the youth in their community. The board works to 
implement tested and effective programs to address the issues and needs that are identified. 


Family Check-Up (also known as Positive Family Support) 





Family Check-Up is a three-tiered intervention implemented in middle schools. The first tier 
involves the establishment of a family resource center in the school and the implementation of a 
six-week prevention curriculum. The second tier is Family Check-Up, an assessment and brief 
motivational interview component for at-risk students. The third tier is the Family Intervention 
Menu, which directs parents of substance-using adolescents to treatment options, parenting 
groups, and family therapy sessions. 


Guiding Good Choices (formerly Preparing for the Drug Free Years) 





Guiding Good Choices is a skills training program for middle school students and their parents 
typically implemented outside normal school hours. The five-session drug resistance and 
education program, implemented one night per week for five weeks, aims to improve parent- 
child interactions that reduce the risk for substance use initiation. 


InShape 


InShape is a college-based brief motivational interviewing intervention that aims to increase 
physical activity, diet, and stress management while reducing substance use through the 
promotion of positive self-image. The program components are typically delivered to young 
adults in a college health clinic setting by a designated fitness specialist. 


keepin’ it REAL 


keepin’ it REAL is a school-based substance use prevention program designed for multicultural 
settings for middle school students. The curriculum is taught by classroom teachers in 45- 
minute sessions once a week for ten weeks and is designed to teach students drug resistance 
skills. 


Life Skills Training 





Life Skills Training is a school-based classroom intervention to reduce the risks of alcohol, 
tobacco, drug abuse, and violence by targeting social and psychological factors associated with 
initiation of risky behaviors. Teachers deliver the program to middle school students in 24 to 30 
sessions over three years. 


Lions Quest Skills for Adolescence 





Lions Quest Skills for Adolescence is a school-based life skills education program designed for 
middle school. The curriculum's 45-minute sessions are designed to prevent substance use and 
bullying behaviors while also teaching anger and stress management skills. 


Mentoring for students: Community-based (including volunteer costs) 





In community-based mentoring programs, volunteer adults are paired with at-risk middle- and 
high-school students to meet weekly at locations of their choosing for relationship building and 
guidance. Community-based organizations provide the adult mentors with training and 
oversight. Mentors are expected to build relationships with mentees with the aim of improving a 
variety of outcomes including crime rates, academic achievement, and substance abuse. This 
analysis includes evaluation findings (in no particular order) for the Washington State Mentors 
program, Big Brothers Big Sisters, Across Ages, Sponsor-a-Scholar, Career Beginnings, the Buddy 
System, and other, locally developed programs. 


Multidimensional Family Therapy (MDFT) for substance abusers 


Multidimensional Family Therapy (MDFT) is an integrative, family-based, multiple systems 
treatment for youth with drug abuse and related behavior problems. The therapy consists of 
four domains: 1) Engage adolescent in treatment, 2) Increase parental involvement with youth 
and improve limit-setting, 3) Decrease family-interaction conflict, and 4) Collaborate with extra- 
familial social systems. Youth are generally aged 11 to 15 and have been clinically referred to 
outpatient treatment. For this meta-analysis, only one study measured the effects of MDFT on 
delinquency and four measured the effects on subsequent substance use. All five studies 
included youth who were referred from the juvenile justice system as well as other avenues. 


Multidimensional Treatment Foster Care 


Multidimensional Treatment Foster Care is an intensive therapeutic foster care alternative to 
institutional placement for adolescents who have problems with chronic antisocial behavior, 
emotional disturbance, and delinquency. Activities include skills training and therapy for youth 
as well as behavioral parent training and support for foster parents and biological parents. 


Project ALERT 


Project ALERT is a middle/junior high school-based program to prevent tobacco, alcohol, and 
marijuana use. Over 11 sessions in the 7th grade and three boosters in the 8th grade, the 
program helps students understand that most people do not use drugs and teaches them to 
identify and resist the internal and social pressures that encourage substance use. 


Project Northland 





Project Northland is a multilevel intervention designed to prevent substance use among 
adolescents in middle school. The 6" grade home component targets parent-child 
communication via homework assignments, group discussions, and the establishment of a 
communitywide task force. The 7" grade school-based curriculum focuses on improving 
resistance skills and social norms regarding teen alcohol use. The 8" grade components include 
the peer-led Powerlines school curriculum, a mock town meeting, and a community action 
project. 


Project STAR 


Also known as the Midwestern Prevention Project, Project STAR is a multi-component 
prevention program with the goal of reducing adolescent tobacco, alcohol, and marijuana use. 
The program consists of a 6th- and 7th-grade intervention supported by parent, community, 
and mass media components addressing the multiple influences of substance use. 


Project SUCCESS 


Project SUCCESS is a school-based prevention program that focuses on high-risk youth. The 
program's four components include: 1) prevention education provided in small groups by a 
professional counselor; 2) individual and group counseling; 3) communications with parents; and 
4) referrals to community agencies. A program counselor is situated in the school throughout 
the academic year. 


Project Towards No Drug Abuse 





Project Towards No Drug Abuse is a prevention program for youth in regular and alternative 
high schools. The curriculum comprises twelve 45-minute lessons implemented in classroom 
settings by teachers or health educators. Using a variety of activities, the program aims to 
increase self-control, communication, decision-making, and motivation to not use substances. 


Project Venture 


Project Venture is a youth development program primarily for 5"- to 8""-grade Native American 
youth. Through the use of outdoor experiential activities such as hiking and camping trips and 
service learning projects, the program attempts to help youth develop social and emotional 
competencies to resist alcohol, tobacco, and other drug use. 
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Red Cliff Wellness School Curriculum 





The Red Cliff Wellness School Curriculum is a school-based substance use curriculum grounded 
in Native American tradition and culture. Designed for students in grades K-12, the curriculum 
aims to reduce risk factors and enhance protective factors related to substance use by 
enhancing core Native American values such as sharing, kindness, honesty, and respect. 


SPORT 





SPORT is a high school-based brief intervention designed to promote a healthy lifestyle via 
improved physical activity, diet, and sleep. Students participate in a 12-minute, one-on-one 
counseling session with a fitness specialist during which they receive a booklet and tailored 
consultation. Students then complete a fitness plan designed to create behavior change and an 
improved self-image. Four weekly fliers that complement the intervention's core content are 
then sent to parents. 


Strengthening Families for Parents and Youth 10-14 





Strengthening Families for Parents and Youth 10-14 (also known as the Iowa Strengthening 
Families Program) is a family-based program that attempts to reduce behavior problems and 
substance use by enhancing parenting skills, parent-child relationships, and family 
communication. The seven-week intervention is designed for 6th-grade students and their 
families. 


Teen Marijuana Check-Up 


Teen Marijuana Check-Up is a brief, school-based intervention for youth meeting diagnostic 
criteria for cannabis use disorders. Youth are introduced to the program via classroom 
presentations. Students receive two 45 to 60 minute motivational interviews a week apart. The 
intervention is provided during the school day without parental involvement. 
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For further information, contact: 
Matt Lemon at 360.586.2744, matt.lemon@wsipp.wa.gov Document No. 14-10-3201 


ie Washington State Institute for Public Policy 


The Washington State Legislature created the Washington State Institute for Public Policy in 1983. A Board of Directors—representing the 
legislature, the governor, and public universities—governs WSIPP and guides the development of all activities. WSIPP’s mission is to 
carry out practical research, at legislative direction, on issues of importance to Washington State. 


